tidystep - rvest网络爬虫

Code

library(rvest)
library(tidyverse)
library(stringr)
library(janitor)
library(priceR)
library(writexl)
library(readxl)
library(clock)
library(lubridate)
library(gt)
library(glue)
library(kableExtra)
library(pander)

用rvest浅爬一下MOM

读入网站

Code

################## www.whiskybase.com
link = "https://www.masterofmalt.com/new-arrivals/whisky-new-arrivals/"
page = read_html(link)

使用selectorgadget选出需要解析需要的内容

下载chrome 插件https://chrome.google.com/webstore/detail/selectorgadget/mhjhnkcfbdhnjickkkdbjoemdmbfginb?hl=en

选中需要的内容，比如这些的名字对应的CSS是[h3 a]

威士忌名字

Code

whisky_name = page %>% html_nodes("h3 a") %>% html_text() 
length(whisky_name)

[1] 25

Code

whisky_name %>% head()

[1] "Tamdhu Quercus Alba Distinction - Release 2"                                                         
[2] "Aberfeldy Tuscan Red Wine Cask 18 Year Old"                                                          
[3] "Culdrain 11 Year Old 2011 (cask 9635499) - The Octave (Duncan Taylor)"                               
[4] "Glen Moray 2013 Amontillado Finish - Warehouse 1"                                                    
[5] "Mortlach 13 Year Old 2008 (casks 314536 & 314537 & 314560) - Un-Chillfiltered Collection (Signatory)"
[6] "Glen Elgin 15 Year Old 2007 - Strictly Limited (Càrn Mòr)"

威士忌购买连接

Code

whisky_link = page %>% html_nodes("h3 a")%>% html_attr("href")
whisky_link %>% head()

[1] "https://www.masterofmalt.com/whiskies/tamdhu/tamdhu-quercus-alba-distinction-release-2-whisky/"                                                   
[2] "https://www.masterofmalt.com/whiskies/aberfeldy/aberfeldy-tuscan-red-wine-cask-18-year-old-whisky/"                                               
[3] "https://www.masterofmalt.com/whiskies/duncan-taylor/culdrain-11-year-old-2011-cask-9635499-the-octave-duncan-taylor-whisky/"                      
[4] "https://www.masterofmalt.com/whiskies/glen-moray/glen-moray-2013-amontillado-finish-warehouse-1-whisky/"                                          
[5] "https://www.masterofmalt.com/whiskies/mortlach/mortlach-13-year-old-2008-casks-314536-314537-314560-un-chillfiltered-collection-signatory-whisky/"
[6] "https://www.masterofmalt.com/whiskies/glen-elgin/glen-elgin-15-year-old-2007-strictly-limited-carn-mor-whisky/"

威士忌图片连接

Code

pic_link=paste0(substring(whisky_link,1, nchar(whisky_link)-1),'.jpg')
pic_link %>% head()

[1] "https://www.masterofmalt.com/whiskies/tamdhu/tamdhu-quercus-alba-distinction-release-2-whisky.jpg"                                                   
[2] "https://www.masterofmalt.com/whiskies/aberfeldy/aberfeldy-tuscan-red-wine-cask-18-year-old-whisky.jpg"                                               
[3] "https://www.masterofmalt.com/whiskies/duncan-taylor/culdrain-11-year-old-2011-cask-9635499-the-octave-duncan-taylor-whisky.jpg"                      
[4] "https://www.masterofmalt.com/whiskies/glen-moray/glen-moray-2013-amontillado-finish-warehouse-1-whisky.jpg"                                          
[5] "https://www.masterofmalt.com/whiskies/mortlach/mortlach-13-year-old-2008-casks-314536-314537-314560-un-chillfiltered-collection-signatory-whisky.jpg"
[6] "https://www.masterofmalt.com/whiskies/glen-elgin/glen-elgin-15-year-old-2007-strictly-limited-carn-mor-whisky.jpg"

威士忌价格，根据爬到的币种转换成人民币价格

Code

price = page %>% html_nodes(".product-box-wide-price") %>% html_text()

hkd_to_cny_rate=exchange_rate_latest("HKD") %>% filter(currency=='CNY')

Daily HKD exchange rate as at end of day 2022-11-13 GMT

Code

price_rmb=round(parse_number(price)*hkd_to_cny_rate$one_hkd_is_equivalent_to)

price_rmb %>% head()

[1] 469 701 561 526 322 631

Code

sources='Mon'

data001=data.frame(whisky_name,price_rmb,whisky_link,pic_link,sources)
data001$bottle=sprintf('![](%s)', data001$pic_link)

Code

data001 %>% head() %>%select(whisky_name,price_rmb,whisky_link)%>% kbl() %>% kable_styling()

whisky_name	price_rmb	whisky_link
Tamdhu Quercus Alba Distinction - Release 2	469	https://www.masterofmalt.com/whiskies/tamdhu/tamdhu-quercus-alba-distinction-release-2-whisky/
Aberfeldy Tuscan Red Wine Cask 18 Year Old	701	https://www.masterofmalt.com/whiskies/aberfeldy/aberfeldy-tuscan-red-wine-cask-18-year-old-whisky/
Culdrain 11 Year Old 2011 (cask 9635499) - The Octave (Duncan Taylor)	561	https://www.masterofmalt.com/whiskies/duncan-taylor/culdrain-11-year-old-2011-cask-9635499-the-octave-duncan-taylor-whisky/
Glen Moray 2013 Amontillado Finish - Warehouse 1	526	https://www.masterofmalt.com/whiskies/glen-moray/glen-moray-2013-amontillado-finish-warehouse-1-whisky/
Mortlach 13 Year Old 2008 (casks 314536 & 314537 & 314560) - Un-Chillfiltered Collection (Signatory)	322	https://www.masterofmalt.com/whiskies/mortlach/mortlach-13-year-old-2008-casks-314536-314537-314560-un-chillfiltered-collection-signatory-whisky/
Glen Elgin 15 Year Old 2007 - Strictly Limited (Càrn Mòr)	631	https://www.masterofmalt.com/whiskies/glen-elgin/glen-elgin-15-year-old-2007-strictly-limited-carn-mor-whisky/

Reference

https://cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html